Assessing the impact of an OSCE-style workshop on internal medicine residents' ability to manage hematologic emergencies: An iterative pilot study Free

Introduction

We conducted a survey of Canadian Internal Medicine (IM) residents which identified significant gaps in hematology education. Of 208 residents surveyed, 84.4% expressed a need for more hematology education in their residency program. A simulation/workshop was perceived to be the most beneficial intervention for learning. This two-iteration pilot study, was designed to assess the feasibility and impact of an OSCE-style workshop on residents' medical knowledge and competence in managing hematologic emergencies.

Methods

The curriculum was delivered in two iterations. In the first iteration, we developed an interactive virtual 1-hour workshop in an OSCE format for IM residents (PGY1-3) rotating at three sites affiliated with the University of Toronto (UofT). Sessions occurred monthly for three months, cumulatively involving 19 trainees. The workshop included two cases, reviewed by faculty experts, based on knowledge gaps identified in the resident survey: microangiopathic hemolytic anemia (MAHA) and acute chest syndrome (ACS). Residents independently completed the cases, followed by a group debrief to review performance and key teaching points. All sessions were taught by a hematology fellow (primary author) who designed the curriculum.

In the second iteration, the workshop was modified to enhance feasibility and to evaluate knowledge retention. It was delivered in the same format as the initial iteration but focused at a single UofT–affiliated hospital to minimize administrative burden and taught by PGY4/PGY5 hematology fellows. Sessions occurred monthly over six months, cumulatively involving 19 trainees. The workshop was adapted to include two additional cases: acute promyelocytic leukemia (APML) and warm autoimmune hemolytic anemia (AIHA).

In both iterations, residents' competence/confidence and medical knowledge were assessed through pre-/post-workshop assessments, retrospective pre-/post-surveys, and debrief feedback. In the second iteration, residents also completed a 1-month post-workshop assessment, with coffee gift cards provided as an incentive to encourage participation.

Wilcoxon signed-rank tests were used to compare responses before, immediately after, and 1 month after the workshop; p<0.05 was considered statistically significant after Holm-Bonferroni correction for multiple comparisons. The study was approved by the UofT Research Ethics Board.

Results

In the first iteration, all participants (19/19) felt that the intervention improved their competence in hematology and provided a unique learning opportunity in a format not otherwise available in their training. Trainees reported improved confidence in diagnosing and managing MAHA and ACS, with p values of <0.05 across all pre/post-Likert scores. With respect to medical knowledge, the pre-workshop mean core was 6.9/10, with a median of 7/10 (interquartile range [IQR] 6-8). The post-workshop assessment mean assessment score was 9.4/10, with a median of 10/10 (IQR 10-10) (p <0.001).

In the second iteration, all survey respondents (13/19) again reported that the intervention subjectively enhanced their competence in hematology. Trainees reported improved confidence in diagnosing and managing MAHA, ACS, APML, and warm AIHA; however, these improvements did not reach statistical significance. With respect to medical knowledge, the pre-workshop mean assessment score for 19 trainees was 7.5/10, with a median of 8.0/10 (IQR 4-10). The immediate post-workshop mean assessment score was 9.3/10, with a median of 9.0/10 (IQR 7-10) (p<0.001). The 1-month post-workshop mean assessment score was 9.3/10, with a median of 9.0/10 (IQR 8-10) (p<0.01 when compared to the pre-assessment score).

Conclusion

This pilot study showed that an OSCE-style virtual workshop is an effective way to enhance internal medicine residents' knowledge and competence in managing hematologic emergencies. While the initial iteration proved educationally effective, logistical challenges limited its feasibility. In the second iteration, targeted modifications, such as consolidating delivery to a single site and recruiting fellows as facilitators, subjectively improved feasibility while maintaining educational efficacy. Residents demonstrated statistically significant improvements in medical knowledge, which were sustained one month post-intervention. This curriculum can serve as a template for other residency programs to implement similar educational interventions for their hematology rotations.